HYbrid Network Engineering Software (HYNES)

A management-plane solution that monitors aggregate traffic (e.g., by reading SNMP MIBs) and decides where to add circuits (network engineering) and/or which IP traffic to reroute (traffic engineering) is not likely to have much value in ESnet because average loading on ESnet links is light (5-10%). Inspite of these light loads, ESnet deployed a dynamic-circuit network called the Science Data Network (SDN) for long flows (“elephant” flows) because of their negative impact on delays experienced by packets from “mice” flows. Unlike passive measurements, such as SNMP link-usage data, which do not show spikes in traffic loads because of multiple-second averaging (10 to 30 sec), active measurement tools such as OneWay Active Measurement Protocol (OWAMP) show these effects. OWAMP is effectively a one-way ping application. Figure 1 shows a 4-hour snapshot of one-way delay measurements obtained with the PerfSONAR OWAMP service. The minimum delay is 28.99 ms, while there are several spikes, some reaching even 60 ms. These are short-lived spikes caused by one or more simultaneous elephant flows. Through experimentation, we determined that in BICand Reno-TCP, if receive buffer sizes are set to values larger than the bandwidth-delay product (BDP), where delay is the round-trip propagation time, as congestion window grows, the number of outstanding segments grows correspondingly. For example, in one experiment, the BDP could hold 100 maximum-sized segments, while the congestion window grew to 1700 segments for a long flow. This means while 100 packets are traversing links, the remaining 1600 packets need to be buffered at en route routers and switches. These cause delay buildups for all flows, thus negatively affecting mice and streamed audio/video flows.